Papers with distributed word representations
Query and Output: Generating Words by Querying Distributed Word Representations for Paraphrase Generation (N18-1)
Copied to clipboard
| Challenge: | Existing models tend to memorize words instead of learning meaning of words . existing models tend not to model semantic information, resulting in incorrect sentences . |
| Approach: | They propose a novel model that generates words by querying distributed word representations . they evaluate model on two paraphrase-oriented tasks, namely text simplification and short abstractive summarization . |
| Outcome: | The proposed model outperforms the baseline model on two paraphrase-oriented tasks . it achieves state-of-the-art performance on these benchmark datasets . |
Subword-level Word Vector Representations for Korean (P18-1)
Copied to clipboard
| Challenge: | Existing research on word vectors for English focuses on decomposing words into subword units and using subwords to improve performance. |
| Approach: | They propose to decompose Korean words into the jamo-level, beyond the character-level . they develop Korean test sets for word similarity and analogy and make them publicly available . |
| Outcome: | The proposed method outperforms word2vec and character-level skip-grams on similarity and analogy tasks and contributes positively toward downstream NLP tasks such as sentiment analysis. |
IsoScore: Measuring the Uniformity of Embedding Space Utilization (2022.findings-acl)
Copied to clipboard
| Challenge: | Several studies suggest that contextualized word embedding models do not isotropically project tokens into vector space. |
| Approach: | They propose to use a tool to measure isotropy to quantify the degree to which a point cloud uniformly utilizes the ambient vector space. |
| Outcome: | The proposed tool is the only available tool that accurately measures how uniformly distributed variance is across dimensions in vector space. |
Word Attribute Prediction Enhanced by Lexical Entailment Tasks (2020.lrec-1)
Copied to clipboard
| Challenge: | a semantic attribute is associated with a designated dimension in attribute-based vector representations . semantic attributes are created by psychological experimental settings involving human annotators . a conceptual attribute of a concept dictates a specific semantic aspect of the concept . |
| Approach: | They propose a two-stage neural network architecture that fine-tunes attribute representations by employing supervised entailment tasks. |
| Outcome: | The proposed method improves performance of semantic/visual similarity/relatedness evaluation tasks. |